Big Data Is Taking a Byte Out of Crime

Credit: Laura Petrilla

The Law Enforcement National Data Exchange helps agencies to better understand their own data, the FBI’s Michael Haas says.

The massive quantities of data that agencies collect is getting bigger every minute, piling into mountains of information. Law enforcement agencies —​​ throughout the federal, state and local levels — are compiling data constantly, but these mountains are useful only if agencies share and apply data effectively.

One way to understand the difference between Big Data as it’s practiced in business and Big Data as it’s applied to law enforcement is to consider the FBI’s Law Enforcement National Data Exchange (N-DEx).

N-DEx is a 10-terabyte data warehouse serving roughly 4,100 law enforcement agencies around the country. “Our mission is to let agencies share their data nationally and allow that data set to come back to them for analysis,” says Michael Haas, unit chief and program manager for N-DEx at the FBI’s Criminal Justice Information Services (CJIS) Division.

But while an online business, for example, can analyze Big Data and take action in real time — such as serving targeted advertising or suggesting relevant products and services — an N-DEx user must do something decidedly old-school before he or she can act on intelligence pieced together from the N-DEx database: pick up a phone.

“In many cases, these are ongoing investigations,” Haas says. “At the end of the day, N-DEx is a tool to encourage and enforce collaboration, and collaboration needs to occur on a human level. There are things officers know that they don’t put on paper. And there are some things on paper that an agency has chosen not to share nationally.”

The human factor aside, N-DEx is a high-tech creature of its time. It is a cloud service, capable of processing criminal information and returning results in fractions of a second over secure Internet links. It is also a Big Data application, comprising roughly 200 million law enforcement records and more than 1.5 billion data points.

“N-DEx is the premier Big Data system today and the most successful implementation of the National Information Exchange Model (NIEM),” says Kshemendra Paul, program manager for the Information Sharing Environment, a governmentwide initiative established by the U.S. Intelligence Reform and Terrorism Prevention Act of 2004 to enable responsible information sharing for national security.

Like many federal data sharing initiatives, N-DEx can trace its origins to 9/11, when it became clear that public safety agencies needed better ways to share threat information. New data standards and systems have since made sharing easier. At the same time, new sources of information — such as social media, video surveillance cameras and sensors — have added to the growing pool of law enforcement information. “Data begets data,” says Ruthbea Clarke, research director for Smart Cities Strategies at IDC Government Insights.

The result: What had been a data sharing challenge has evolved into a Big Data opportunity. Broadly speaking, Big Data solutions for law enforcement fall into two categories: correlation analysis and predictive policing.

Big Data for Decision-Making

“The use of analytics has been one of the most significant movements across law enforcement nationwide,” says Steve Ambrosini, executive director of the IJIS Institute, a nonprofit organization that works with technology companies and state, local, tribal and federal agencies to promote information exchange initiatives. The IJIS Institute is funded by its member organizations and through grants from the Department of Justice, the Bureau of Justice Assistance and the Department of Homeland Security.

“Big Data theory is playing its biggest role through the value of optimized decision-making, the persistent availability of all forms of data and the ability to streamline sharing through standards,” Ambrosini says. “The challenges are more basic. We’re just starting to look at governance and standards for the actual use of Big Data. There are a lot of rules for what you can do with this stuff that are very different from the business domain.”

When it comes to N-DEx, Haas is quick to point out that CJIS acts only as a steward of the data. Contributing law enforcement agencies do, in fact, use NIEM and Logical Entity eXchange Specifications to upload data into the N-DEx data warehouse for rapid centralized processing, but the latest information stays with the local agency, and none of it becomes the property of the FBI. “This allows agencies to manage how their information is shared and disseminated with different disciplines in the criminal justice community while still retaining control and ownership of the information locally,” Haas says.

Download our free white paper.

N-DEx contributing agencies follow specific data handling and tagging guidelines. They use a simple traffic light model to indicate permissions: Green means anyone on N-DEx is allowed to see the information; yellow indicates a point of contact at the data-owning agency for accessing relevant information; and red means the information is not available to N-DEx users.

Why would agencies upload data to N-DEx and not make it visible to other users? Haas says it’s so agencies can exploit the system’s search and analysis tools. In this respect, N-DEx is a robust cloud service.

“N-DEx allows agencies to understand things about their own data that they can’t understand,” Haas says. “Some agencies decide to share an extremely large amount of the information with the N-DEx system but allow visibility only to their own agency. It allows them to link and connect records within their own agency data in a way that their own systems may not be capable of.”

Users mine the data in N-DEx in two primary ways. First, by using a straight forward text search engine: Through a web interface that CJIS provides, agencies can search records and use keywords to scour narrative, unstructured information. The second N-DEx search engine performs correlation and resolution analysis.

N-DEx records generally are what users might expect — several pages of a police report, for example, with descriptions and maybe a picture or two. Within each record are what CJIS calls entities — people, places, things or events — that are tagged by the originating agency according to an Information Exchange Package Documentation published by the Office of Justice Programs Clearinghouse. N-DEx employs proprietary algorithms to find relationships among entities.

“This is the engine that can tell that a person identified in Record A by law enforcement Agency B is really the same person we encountered two years ago through another agency,” says Joseph Sheets, lead developer for N-DEx operations and management.

Once an N-DEx user has detected a person of interest, the system offers several built-in ways of visualizing relevant data.

“You can basically press a button and get a composite biography,” Haas says. “Press another button to get a geographical view of where things happened, another button will give you a link analysis of all the person’s relationships, and another will give you a timeline of the person’s different encounters.”

CJIS offers the N-DEx system free to agencies online, but recognizes that many have their own analysis and correlation tools.

Predictive Policing

A more nascent and cutting-edge use of Big Data in law enforcement is predictive policing — the practice of analyzing information to identify crimes before they happen. “The holy grail is prediction and prevention,” says IDC’s Clarke.

The Smart Policing Initiative (SPI), a consortium led by the Bureau of Justice Assistance (BJA), offers grants to local agencies in support of programs that leverage data in law enforcement. Since 2009, the SPI has funded 35 projects.

“There’s a real desire among police executives to use data and science to test out new strategies and find more effective ways of addressing crime,” says Denise O’Donnell, director of the BJA.

O’Donnell says agencies in the SPI program have used Big Data to analyze issues such as prescription drug abuse, violent crime hot spots, spikes in convenience store robberies, correlations between home burglaries and truancy, and the effectiveness of foot patrols compared with police cars. In Cambridge, Mass., one SPI-funded program is using analytics algorithms created at the Massachusetts Institute of Technology to take known information about violent offenders and not only apply it to unsolved crimes, but also use it to predict where future crimes might occur.

“In many jurisdictions, agencies are trying to police more effectively by focusing more on places than on people,” says Ambrosini.

Serious Progress

Several years ago, the Bureau of Indian Affairs (BIA), which provides direct law enforcement services to 26 reservations across the country, was facing particularly high violent crime rates on several reservations. The BIA chose four reservations to implement a high-priority performance goal to reduce violent crime by 5 percent in a 24-month period. It pulled together statistics and information from crime reports into a central repository and began performing trend analysis on the data.

“We were able to reduce violent crime on those reservations by 35 percent through intelligence-led policing,” says Charles Addington, associate director of field operations in the BIA’s Office of Justice Services. “By getting the data into this initial system to figure out patterns of where crimes were occurring and when, local law enforcement officials were able to put their resources where they needed to be.”

The Bureau of Indian Affairs has since rolled out its crime trend analysis system to additional reservations, but Addington says it is now also in the process of formatting the data for integration with the Department of Interior’s new Incident Management, Analysis and Reporting System.

Addington also says he’s been meeting with the FBI’s CJIS group to begin sharing data via N-DEx.

Ambrosini calls current initiatives to use Big Data in law enforcement “embryonic.” IDC’s Clarke sees a future in which automation presents new information — and issues.

“There’s a lot that agencies could automate,” she says. “Sensors can provide situational awareness and send streams of data back to operation centers. These are examples of Big Data that don’t necessarily rely on data governance. But when we automate data gathering, it raises new issues, like whether the data is good or what’s an appropriate automated reaction?”

Whatever the issues — technology, policy or governance — Ambrosini expects rapid advancement because of what’s at stake: public safety. “We’re going to see new Big Data developments in law enforcement because no one is going to want to get in the way of them.”